医疗关系提取（MRE）任务旨在提取医学文本中实体之间的关系。传统的关系提取方法通过探索句法信息，例如依赖树。但是，由外域解析器产生的医学文本的1好的依赖树的质量相对有限，因此医疗关系提取方法的性能可能会退化。为此，我们提出了一种基于因果解释理论的医学文本中共同模拟语义和句法信息的方法。我们生成依赖性森林，这些森林由1-最佳依赖树组成。然后，采用特定于任务的因果解释者来修剪依赖性森林，该森林将进一步送入设计的图形卷积网络，以学习下游任务的相应表示。从经验上讲，基准医学数据集的各种比较证明了我们模型的有效性。

很少有学习模型学习人类注释有限，而这种学习范式在各种任务中证明了实用性数据使该模型无法充分探索语义信息。为了解决这个问题，我们将知识蒸馏引入了几个弹出的对象检测学习范式。我们进一步进行了激励实验，该实验表明，在知识蒸馏的过程中，教师模型的经验误差将少数拍物对象检测模型的预测性能（作为学生）退化。为了了解这种现象背后的原因，我们从因果理论的角度重新审视了几个对象检测任务上知识蒸馏的学习范式，并因此发展了一个结构性因果模型。遵循理论指导，我们建议使用基于后门调整的知识蒸馏方法，用于少数拍物检测任务，即Disentangle和Remerge（D＆R），以对相应的结构性因果模型进行有条件的因果干预。从理论上讲，我们为后门标准提供了扩展的定义，即一般后门路径，可以在特定情况下扩展后门标准的理论应用边界。从经验上讲，多个基准数据集上的实验表明，D＆R可以在几个射击对象检测中产生显着的性能提升。

时间序列分析已在网络安全，环境监测和医学信息学等不同应用中取得了巨大成功。在不同时间序列之间学习相似性是一个关键问题，因为它是下游分析（例如聚类和异常检测）的基础。由于事件触发的传感产生的时间序列的复杂时间动态，通常不清楚哪种距离度量适合相似性学习，这在各种应用中很常见，包括自动驾驶，交互式医疗保健和智能家庭自动化。本文的总体目标是开发一个无监督的学习框架，该框架能够在未标记的事件触发时间序列中学习任务感知的相似性。从机器学习有利位置，提出的框架可以利用层次多尺度序列自动编码器和高斯混合模型（GMM）的功能，以有效地学习时间序列的低维表示。最后，可以轻松地将获得的相似性度量可视化以进行解释。拟议的框架渴望提供一块垫脚石，从而产生一种系统的模型方法，以在许多事件触发的时间序列中学习相似之处。通过广泛的定性和定量实验，揭示了所提出的方法的表现大大优于最先进的方法。

多尺度特征层次结构已在计算机视觉区域的成功中得到了见证。这进一步激发了研究人员设计自然语言处理的多尺度变压器，主要是基于自我发项机制。例如，限制跨头部的接收场或通过卷积提取局部细粒度特征。但是，大多数现有作品都直接建模了本地功能，但忽略了单词边界信息。这导致了缺乏解释性的多余和模棱两可的注意力分布。在这项工作中，我们在不同的语言单元中定义了这些量表，包括子字，单词和短语。我们通过基于单词边界信息和短语级别的先验知识之间建立量表之间的关系来构建多尺度变压器模型。提出的\ textbf {u} niversal \ textbf {m} ulti \ textbf {s} cale \ textbf {t} ransformer，即在两个序列生成任务上评估。值得注意的是，它在几个测试组上的强大基线上产生了一致的性能，而无需牺牲效率。

Embedding words in vector space is a fundamental first step in state-of-the-art natural language processing (NLP). Typical NLP solutions employ pre-defined vector representations to improve generalization by co-locating similar words in vector space. For instance, Word2Vec is a self-supervised predictive model that captures the context of words using a neural network. Similarly, GLoVe is a popular unsupervised model incorporating corpus-wide word co-occurrence statistics. Such word embedding has significantly boosted important NLP tasks, including sentiment analysis, document classification, and machine translation. However, the embeddings are dense floating-point vectors, making them expensive to compute and difficult to interpret. In this paper, we instead propose to represent the semantics of words with a few defining words that are related using propositional logic. To produce such logical embeddings, we introduce a Tsetlin Machine-based autoencoder that learns logical clauses self-supervised. The clauses consist of contextual words like "black," "cup," and "hot" to define other words like "coffee," thus being human-understandable. We evaluate our embedding approach on several intrinsic and extrinsic benchmarks, outperforming GLoVe on six classification tasks. Furthermore, we investigate the interpretability of our embedding using the logical representations acquired during training. We also visualize word clusters in vector space, demonstrating how our logical embedding co-locate similar words.

Depression is a leading cause of death worldwide, and the diagnosis of depression is nontrivial. Multimodal learning is a popular solution for automatic diagnosis of depression, and the existing works suffer two main drawbacks: 1) the high-order interactions between different modalities can not be well exploited; and 2) interpretability of the models are weak. To remedy these drawbacks, we propose a multimodal multi-order factor fusion (MMFF) method. Our method can well exploit the high-order interactions between different modalities by extracting and assembling modality factors under the guide of a shared latent proxy. We conduct extensive experiments on two recent and popular datasets, E-DAIC-WOZ and CMDC, and the results show that our method achieve significantly better performance compared with other existing approaches. Besides, by analyzing the process of factor assembly, our model can intuitively show the contribution of each factor. This helps us understand the fusion mechanism.

Model bias triggered by long-tailed data has been widely studied. However, measure based on the number of samples cannot explicate three phenomena simultaneously: (1) Given enough data, the classification performance gain is marginal with additional samples. (2) Classification performance decays precipitously as the number of training samples decreases when there is insufficient data. (3) Model trained on sample-balanced datasets still has different biases for different classes. In this work, we define and quantify the semantic scale of classes, which is used to measure the feature diversity of classes. It is exciting to find experimentally that there is a marginal effect of semantic scale, which perfectly describes the first two phenomena. Further, the quantitative measurement of semantic scale imbalance is proposed, which can accurately reflect model bias on multiple datasets, even on sample-balanced data, revealing a novel perspective for the study of class imbalance. Due to the prevalence of semantic scale imbalance, we propose semantic-scale-balanced learning, including a general loss improvement scheme and a dynamic re-weighting training framework that overcomes the challenge of calculating semantic scales in real-time during iterations. Comprehensive experiments show that dynamic semantic-scale-balanced learning consistently enables the model to perform superiorly on large-scale long-tailed and non-long-tailed natural and medical datasets, which is a good starting point for mitigating the prevalent but unnoticed model bias.

Tsetlin Machine (TM) has been gaining popularity as an inherently interpretable machine leaning method that is able to achieve promising performance with low computational complexity on a variety of applications. The interpretability and the low computational complexity of the TM are inherited from the Boolean expressions for representing various sub-patterns. Although possessing favorable properties, TM has not been the go-to method for AI applications, mainly due to its conceptual and theoretical differences compared with perceptrons and neural networks, which are more widely known and well understood. In this paper, we provide detailed insights for the operational concept of the TM, and try to bridge the gap in the theoretical understanding between the perceptron and the TM. More specifically, we study the operational concept of the TM following the analytical structure of perceptrons, showing the resemblance between the perceptrons and the TM. Through the analysis, we indicated that the TM's weight update can be considered as a special case of the gradient weight update. We also perform an empirical analysis of TM by showing the flexibility in determining the clause length, visualization of decision boundaries and obtaining interpretable boolean expressions from TM. In addition, we also discuss the advantages of TM in terms of its structure and its ability to solve more complex problems.

The stock market prediction has been a traditional yet complex problem researched within diverse research areas and application domains due to its non-linear, highly volatile and complex nature. Existing surveys on stock market prediction often focus on traditional machine learning methods instead of deep learning methods. Deep learning has dominated many domains, gained much success and popularity in recent years in stock market prediction. This motivates us to provide a structured and comprehensive overview of the research on stock market prediction focusing on deep learning techniques. We present four elaborated subtasks of stock market prediction and propose a novel taxonomy to summarize the state-of-the-art models based on deep neural networks from 2011 to 2022. In addition, we also provide detailed statistics on the datasets and evaluation metrics commonly used in the stock market. Finally, we highlight some open issues and point out several future directions by sharing some new perspectives on stock market prediction.

To improve the performance of the dual-encoder retriever, one effective approach is knowledge distillation from the cross-encoder ranker. Existing works construct the candidate passages following the supervised learning setting where a query is paired with a positive passage and a batch of negatives. However, through empirical observation, we find that even the hard negatives from advanced methods are still too trivial for the teacher to distinguish, preventing the teacher from transferring abundant dark knowledge to the student through its soft label. To alleviate this issue, we propose ADAM, a knowledge distillation framework that can better transfer the dark knowledge held in the teacher with Adaptive Dark exAMples. Different from previous works that only rely on one positive and hard negatives as candidate passages, we create dark examples that all have moderate relevance to the query through mixing-up and masking in discrete space. Furthermore, as the quality of knowledge held in different training instances varies as measured by the teacher's confidence score, we propose a self-paced distillation strategy that adaptively concentrates on a subset of high-quality instances to conduct our dark-example-based knowledge distillation to help the student learn better. We conduct experiments on two widely-used benchmarks and verify the effectiveness of our method.